Data Mining using Genetic Programming: The Implications of Parsimony on Generalization Error

A common data mining heuristic is, “When choosing between models with the same training error, less complex models should be preferred as they perform better on unseen data.” This heuristic may not always hold. In genetic programming a preference for less complex models is implemented as i) placing a limit on the size of the evolved program ii) penalizing more complex individuals, or both. This paper presents a GPvariant with no limit on the complexity of the evolved program that generates highly accurate models on a common dataset.